December 6, 2005

Let’s do something empirical to back up the rational…

Hey,

I am working with a grad student who wants to get a quick study up and
running. I thought of a study where we compare information retrieval times
and errors when people are using 2 kinds of info organizations: traditional
hierarchical/taxonomic categories versus tags.

Between groups: We are thinking of having subjects come in and sort 100
photographs into categories or tag 100 pictures. Then, a few days or one
week later, they would have to retrieve a subset of those pictures. Of
course we would control the amount of time spent on the pictures.

What do you think?

Rich

* Filed by prof_chuck at 3:19 pm under Uncategorized
* 6 Comments

Create a free edublog to get your own comment avatar (and more!)

6 Responses to “Let’s do something empirical to back up the rational…”

  1.   prof_chuck Says:
    December 6th, 2005 at 3:22 pm

    I’ll play devil’s advocate, but I won’t ask any stupid questions ;-) You don’t have PTSD do you?

    —————————————————

    First, great idea, collect multiple DVs on memory/performance with 100 photos (perhaps randomly selected from Flickr). Two potential IV manipulations are a sorting task (bookmarks into categories) and tags.

    What will the directions be like? What are the classic sorting directions? What will be the tagging directions? (Flickr directions / del.icio.us directions / directions controlled or matched to the sorting directions)

    We are all obviously thinking that tagging will lead to better recall/faster retrieval time/fewer errors, but why? I think I know. Ask your student?

    What additional directions could be added to the sorting task to make it a tagging task?

    Don’t forget to measure and see if any participants are expert taggers.

    This could easily be a paper and pencil task. Are you going to write your own program/environment or use some standard(s)?

    So is this really a sorting task in the classic sense or are you going to have everyone create folders and sort like they bookmark (no super-ordinate or subordinate folders).

    Is the task of tagging something that creates more of an episodic memory and the category task more semantic. How do both systems contribute to the process. How does procedural memory contribute to the process(experts/automated)

    Feel free to contact me or have your student contact me to bounce ideas around.

  2.   prof_chuck Says:
    December 6th, 2005 at 3:27 pm

    Why Rich and I were email is LOL ridiculous. Thanks rich for pointing that out. From here on out we blog it. Welcome to our lab meeting!

    …and rich replies:

    Wow, great comments:

    Right now, we are collecting 100 random photos from istockphoto.com. For
    the task, I was going to program something but it seems very difficult.
    Another difficulty is the retrieval phase. How exactly are they going to
    retrieve the pictures in each condition. In the sorting condition, that
    seems simple. But in the tagging condition, how do they retrieve a picture?
    Clicking on a tag cloud?

    Some thoughts were that tagging involves more engagement with the material
    (higher levels of processing) than sorting in folders. Additionally, the
    act of creating a tag may lead to a more durable memory trace (generation
    effect). And probably the weakest support for tagging is that it mirrors
    the semantica organization or LTM. This is your area, do these reasons
    sound logical, if not a little hazy.

    In a small sense, the sorting task is like a tagging task because the user
    is naming the pile of pictures.

    Paper and pencil was the initial plan (giving users a stack of 100 photos).
    But how would tagging happen?

    The sorting task was going to be like how they bookmark; or organize
    computer files. As many levels as you want.

    “Is the task of tagging something that creates more of an episodic
    memory and the category task more semantic. How do both systems
    contribute to the process. How does procedural memory contribute to
    the process(experts/automated)”

    What a great question…i hadn’t thought about the episodic/semantic
    distinction.

  3.   dr. pak Says:
    December 6th, 2005 at 4:05 pm

    Wow, fancy site…now this is collaboration!

  4.   Piers Says:
    December 15th, 2005 at 4:22 pm

    Looks interesting! It’s vaguely related but you might want to have a look here : http://www.publictechnology.net/modules.php?op=modload&name=News&file=article&sid=2681

    One thing – will you be testing people on tags others have given photos?

  5.   Christina Pikas Says:
    December 16th, 2005 at 8:38 pm

    Sounds interesting… wrt Piers’ comment, I think it’s important that the searchers are not the taggers… That could definitely skew. My other comment is that I’d like to see the hierarchical system be a professionally prepared system (so terms are orthogonal, scope notes to describe where things go…) — otherwise, it’s pretty much the same thing as tagging, right? Find classification system for pictures and have your subjects classify pictures according to that scheme… IMHO

  6.   Paul Andersen Says:
    December 22nd, 2005 at 11:42 pm

    I also agree that the searchers should not be the taggers if you want a good experimentation. Are you going to have the taggers pick from a controlled vocabulary or simply make up their own tags? It would be interesting to do both, and perhaps try two different vocabularies: one which has been created socially via something like del.icio.us, and one that might not offer as many alternatives, like dmoz.org. It would be interesting to see if virtual peer pressure is a factor.